Motion estimation is an effective inter-frame coding technique to exploit temporal redundancy in video sequences. Motion-compensated inter-frame coding has been widely used in various international video coding standards. For video coding systems incorporating inter-frame or inter-picture coding, the processing of a current picture can utilize information from previously coded pictures (or slices or regions). These previously coded pictures are decoded before they are used for prediction and the decoded pictures are called ‘reference pictures’. In newer video coding standards, such as Advanced Vide Coding (AVC, also known as H.264) and emerging High Efficiency Video Coding (HEVC), inter-picture redundancy has been well exploited by using very elaborate reference pictures. The reference pictures have to be stored in reference picture buffer in the encoder side as well as the decoder side. Since the number of reference pictures used may be quite large, it becomes a critical issue for a coding system to manage reference pictures efficiently.
In the HEVC standard (Bross et al, “High Efficiency Video Coding (HEVC) text Specification Draft 9”, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, JCTVC-K1003_v10, October 2012.), flexible reference picture management is disclosed. The encoder can choose the reference parameters for coding each picture. The reference parameters may include the number of reference pictures used to construct the reference lists, the order of the reference pictures in the reference lists and reference picture usage. The reference picture information is transmitted in the video bitstream so that the decoder can recover the corresponding reference pictures at the decoder side. To implement these functions, reference picture management design according to HEVC is described as follows.
A POC (Picture Order Count) is assigned to each picture and conveyed in the slice header from the encoder to the decoder. POC represents the output order (i.e., display order) of a picture. For a slice, a Reference Picture Set (RPS) is used to define the reference pictures used for the current slice. In RPS, reference pictures are located by the difference between POC of the current picture and the reference picture. The syntax of short-term RPS based on POC according to HEVC is shown in FIG. 1. Since the output order may be different from the coding order, POCs of the reference pictures may be smaller or larger than the POC of the current picture. There are two ways, i.e., direct and predictive modes, to code the POCs according to HEVC as indicated by inter_ref_pic_set_prediction_flag. If the flag is equal to 0, the reference picture set of the current picture is represented using the direct mode. If the flag is equal to 1, the reference picture set of the current picture is represented using the predictive mode. In the direct mode, POCs of reference pictures are coded as set s0 corresponding to POCs lower than the current picture and set s1 corresponding to POCs larger than the current picture respectively. These reference pictures may be used by the current picture if used_by_curr_pic_sx_flag is equal to 1, or used by following pictures but not used by the current picture if used_by_curr_pic_sx_flag is equal to 0. In the predictive mode, POCs of reference pictures are predicted by a previously coded RPS indicated by delta_idx_minus1.
RPS can be incorporated in SPS (Sequecne Parameter Set) as dipicted in FIG. 2. Multiple reference picture sets can be transmitted in SPS, as indicated by num_short_term_ref_pic_sets. A slice may select a RPS transmitted in SPS as indicated by short_term_ref_pic_set_idx or the slice may use a new RPS transmitted in the slice header. The corresponding syntax structure in the slice header is shown in FIG. 3.
For a decoded picture in DPB (Decoded Picture Buffer), if the picture is not included in the RPS of the current picture, the picture is not a reference picture any more and can be removed from the DPB after output.
Reference lists can be constructed after reference pictures used by the current picture are specified. A default list construction rule for HEVC is defined as shown in FIG. 4. For List 0, reference pictures with POCs lower than the current picture in the decending POC order are placed in DPB from the top of the DPB buffer list. Reference pictures with POCs higher than the current picture are then placed in the DPB according to the accending POC order. Accordingly, List 0 for the example of FIG. 4 is 2, 1, 4 and 5. For List 1, reference pictures with POCs higher than the current picture are placed in the DPB from the top of the DPB buffer list according to the accending POC order. Reference pictures with POCs lower than the current picture are then placed in the DPB according to the dcending POC order. Accordingly, List 1 for the example of FIG. 4 is 4, 5, 2 and 1. The encoder can also assign the reference lists in an explicit way by sending a ref_pic_lists_modification( ) syntax table in the slice header as shown in FIG. 5.